-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add unique constraints to the email and username fields in the galaxy_user table #18493
Conversation
Failures are relevant: sqlite and postgres create different db objects based on the column specification |
|
||
def upgrade(): | ||
with transaction(): | ||
create_unique_constraint(constraint_name, table_name, [column_name]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is going to fail without #18492. I would pull that into the migration. Once the constraint is present we'll never need the script, so I don't see why we have it live as a separate script.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My initial thinking was that we could run that script on the current release, which would deduplicate the usernames and, with that, stop the errors. However, the more I thought about it, I became convinced that it shouldn't be merged into a stable release: the script is a feature and belongs in dev. So yes, I'll move it into the same PR.
That said, I'm not sure we should be combining database schema revisions with data migrations in the same revision script. I think it may be better to have a script and reference it in the db migration module as an "upgrade note" (like I did here) + in the release admin notes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should be pragmatic here and not provide a migration that can fail, which is going to be a major pain for admins. This is not a huge amount of data that needs to be altered, we're talking a bit more than 200 users on main.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK. I suppose we can distinguish between (a) a data migration that addresses business logic needs and (b) a data migration that fixes inconsistent data to enable the db schema migration; and combine the latter with the db schema migration into one revision script. We should then mention such cases in the release admin notes to alert admins that some data is being changed (adding a label now).
c57cb7d
to
48bf21f
Compare
Implementation note on indexes and unique constraints If we are adding a unique constraint to a column that had an index defined on it, we must drop that index, because it did not enforce a unique constraint. However, only existing databases will contain that index, new databases will not - so we must check for its existence in the migration script before issuing the |
48bf21f
to
9edce5e
Compare
unit test failures relevant: the tests don't respect the new constraints. I'll fix this. |
beac130
to
0c43588
Compare
0c43588
to
600523b
Compare
So this has a new kind of test: individual migration test. The purpose is to test how existing data (that would prevent the execution of a particular migration) is fixed in the database during the migration process. For example, let's say we are adding a unique constraint to a field that currently contains duplicate values. We must cleanup the data before we can run this migration, so we have a script that handles this cleanup (by deleting duplicates, or deduplicating them, etc.). This test verifies such a script. Here's what the test does:
|
Integration test failures unrelated. |
Ref #18487
BEFORE MERGING:
Migrations that add unique constraints to email and username fields in the galaxy_user table + model definition update.
Unique constraints are implemented with indexes in both postgres and sqlite, so the 2 existing indexes are no longer needed - which is why the migration drops them.
The model definitions for both tables contain both
index=True
andunique=True
: that's not a problem. Havingunique=True
would be sufficient, but this, I think, makes it a little easier to understand (maybe?). I've tested manually - in any case a b-tree index is created with a unique constraint.How to test the changes?
(Select all options that apply)
License